Purpose

To remove the constraint that keypoints cannot be transformed using flips.

Specification

Using albumentations>2.0.0 and testing using the luxonis_ml data inspect [dataset] --aug-config [augmentations] as well as using the debugger to manually check transformations before and after, the effect of horizontally or vertically augmenting an image and its keypoints works correctly and does not change the index of the keypoints. The tests were performed on the COCO tiger dataset.

Dependencies & Potential Impact

Albumentations>=2.0.0 (already in luxonis-ml/data/requirements.txt)

Deployment Plan

Testing & Validation

Test 1: flips by saving the result of the keypoints before and after vertical flip. In this case, width = 1280 and height = 720:

Before:

[[988. 356. 2.], [933. 229. 2.], [766. 177. 2.], [319. 194. 2.], [360. 476. 2.], [395. 566. 2.], [322. 566. 2.], [292. 453. 2.], [806. 378. 2.], [863. 527. 2.], [632. 416. 2.], [584. 545. 2.]]

After:

[[988. 363. 2.], [933. 490. 2.], [766. 542. 2.], [319. 525. 2.], [360. 243. 2.], [395. 153. 2.], [322. 153. 2.], [292. 266. 2.], [806. 341. 2.], [863. 192. 2.], [632. 303. 2.], [584. 174. 2.]]

As you can see, the keypoints after the transformation have the same x coordinate, and their y coordinate is (720 - after_y) = before_y.

Testing on images using luxonis_ml data inspect:

Original image with keypoints

Vertical flip

Mosaic augmentation

MixUp

Horizontal flip

As can be seen, the relevant keypoint indices stay the same despite the transformation. For example, index 2 is still the nose, and indices 5, 6, 9 and 11 are still the feet, even if the order of the location of the keypoints changes after transformations.